9 research outputs found
Anisotropic Proximal Gradient
This paper studies a novel algorithm for nonconvex composite minimization
which can be interpreted in terms of dual space nonlinear preconditioning for
the classical proximal gradient method. The proposed scheme can be applied to
composite minimization problems whose smooth part exhibits an anisotropic
descent inequality relative to a reference function. In the convex case this is
a dual characterization of relative strong convexity in the Bregman sense. It
is proved that the anisotropic descent property is closed under pointwise
average if the dual Bregman distance is jointly convex and, more specifically,
closed under pointwise conic combinations for the KL-divergence. We analyze the
method's asymptotic convergence and prove its linear convergence under an
anisotropic proximal gradient dominance condition. This is implied by
anisotropic strong convexity, a recent dual characterization of relative
smoothness in the Bregman sense. Applications are discussed including
exponentially regularized LPs and logistic regression with nonsmooth
regularization. In the LP case the method can be specialized to the Sinkhorn
algorithm for regularized optimal transport and a classical parallel update
algorithm for AdaBoost. Complementary to their existing primal interpretations
in terms of entropic subspace projections this provides a new dual
interpretation in terms of forward-backward splitting with entropic
preconditioning
Sublabel-Accurate Relaxation of Nonconvex Energies
We propose a novel spatially continuous framework for convex relaxations
based on functional lifting. Our method can be interpreted as a
sublabel-accurate solution to multilabel problems. We show that previously
proposed functional lifting methods optimize an energy which is linear between
two labels and hence require (often infinitely) many labels for a faithful
approximation. In contrast, the proposed formulation is based on a piecewise
convex approximation and therefore needs far fewer labels. In comparison to
recent MRF-based approaches, our method is formulated in a spatially continuous
setting and shows less grid bias. Moreover, in a local sense, our formulation
is the tightest possible convex relaxation. It is easy to implement and allows
an efficient primal-dual optimization on GPUs. We show the effectiveness of our
approach on several computer vision problems
Convex relaxations for large-scale graphically structured nonconvex problems with spherical constraints: An optimal transport approach
In this paper we derive a moment relaxation for large-scale nonsmooth
optimization problems with graphical structure and spherical constraints. In
contrast to classical moment relaxations for global polynomial optimization
that suffer from the curse of dimensionality we exploit the partially separable
structure of the optimization problem to reduce the dimensionality of the
search space. Leveraging optimal transport and Kantorovich--Rubinstein duality
we decouple the problem and derive a tractable dual subspace approximation of
the infinite-dimensional problem using spherical harmonics. This allows us to
tackle possibly nonpolynomial optimization problems with spherical constraints
and geodesic coupling terms. We show that the duality gap vanishes in the limit
by proving that a Lipschitz continuous dual multiplier on a unit sphere can be
approximated as closely as desired in terms of a Lipschitz continuous
polynomial. The formulation is applied to sphere-valued imaging problems with
total variation regularization and graph-based simultaneous localization and
mapping (SLAM). In imaging tasks our approach achieves small duality gaps for a
moderate degree. In graph-based SLAM our approach often finds solutions which
after refinement with a local method are near the ground truth solution
Discrete-Continuous ADMM for Transductive Inference in Higher-Order MRFs
This paper introduces a novel algorithm for transductive inference in
higher-order MRFs, where the unary energies are parameterized by a variable
classifier. The considered task is posed as a joint optimization problem in the
continuous classifier parameters and the discrete label variables. In contrast
to prior approaches such as convex relaxations, we propose an advantageous
decoupling of the objective function into discrete and continuous subproblems
and a novel, efficient optimization method related to ADMM. This approach
preserves integrality of the discrete label variables and guarantees global
convergence to a critical point. We demonstrate the advantages of our approach
in several experiments including video object segmentation on the DAVIS data
set and interactive image segmentation
Dualities for Non-Euclidean Smoothness and Strong Convexity under the Light of Generalized Conjugacy
Relative smoothness and strong convexity have recently gained considerable attention in optimization. These notions are generalizations of the classical Euclidean notions of smoothness and strong convexity that are known to be dual to each other. However, conjugate dualities for non-Euclidean relative smoothness and strong convexity remain an open problem, as noted earlier by Lu, Freund, and Nesterov [SIAM J. Optim., 28 (2018), pp. 333–354]. In this paper, we address this question by introducing the notions of anisotropic strong convexity and smoothness as the respective dual counterparts. The dualities are developed under the light of generalized conjugacy, which leads us to embed the anticipated dual notions within the superclasses of certain upper and lower envelopes. In contrast to the Euclidean case, these inclusions are proper in general, as showcased by means of counterexamples